DISTRIBUTION  STATEMENT  A.  Approved  for  public  release;  distribution  is  unlimited 


Drifter  Motion  Planning  for  Optimal  Surveillance  of  the  Ocean 

Progress  Report 

Stochastic  Prediction  and  Control  in  Multiscale  Systems 

PI:  Dr.  Ira  B.  Schwartz 
Naval  Research  Laboratory 
Code  6792 

Washington,  DC  20375 
Ira.Schwartz@nrl.navy.mil 

Award  Number:  N0001408WX20780 


PROBLEM  AND  APPROACH 

The  current  problems  of  the  proposal  deal  with  littoral  monitoring  of  large  regions  of  the  ocean  using 
single  or  multiple  gliders.  This  portion  of  the  joint  proposal  with  UCSB  deals  with  the  stochastic 
modeling  and  control  of  gliders  in  the  ocean  environment.  Basically,  the  goals  are  to  model  stochastic 
flows  on  gliders,  considered  as  particles  forced  by  the  ocean,  and  design  optimal  control  to  keep  the 
gliders  in  a  specific  region.  That  is,  the  control  will  be  designed  to  increase  loitering  time  in  a  specified 
region.  Without  controls,  the  stochastic  forcing  of  the  environment  will  cause  the  gliders  to  leave  the 
region  of  interest  in  finite  time.  Controls  were  designed  to  optimize  the  lifetime  by  minimizing  the 
actuation  intervals. 

RESEARCH  TASKS  AND  OVERALL  PROGRESS 

a.  Here  we  designed  a  model  of  a  glider  on  the  ocean  surface  which  is  acted  upon  by  stochastic 
forcing.  We  modeled  the  glider  as  a  point  particle,  and  considered  a  multiscale  flow  in  time.  Based  on 
the  model  of  the  flow,  we  are  able  to  analytically  predict  the  probability  of  escape,  and  the  mean 
lifetimes  in  a  finite  region.. 

b.  New  mathematical  theories  of  stochastic  model  reduction  were  developed  to  analyze  the  probability 
of  escape.  Using  these  new  tools,  an  optimal  path  to  escape  can  be  identified.  Using  this  path,  optimal 
stochastic  controls  were  derived  to  extend  the  lifetime  of  the  specified  region,  thereby  enhancing 
loitering  times. 

c.  Multi-particle  in  stochastic  environments:  We  considered  the  effect  of  communication  delay,  or 
latency,  between  multiple  sensing  platforms  in  stochastic  flows.  Here  we  showed  how  latency  may 
destabilize  the  sensing  swarm  so  that  a  particular  patterns  used  for  monitoring  a  given  area  are  no 
longer  stable. 

d.  Using  networked  local  measurements  to  achieve  global  measurements;  Here  we  used  multiple 
sensors  with  communication  to  determine  the  large  region  surface  temperature  of  the  ocean. 


1 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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NONLINEAR  STOCHASTIC  MODEL-MULTIPLE  SCALES  IN  A  SINGLE 
LAYER  OCEAN  MODEL 


We  developed  a  model  of  the  wind  forced  ocean  in  a  single  layer.  The  model  takes  into  account 
multiple  scales  and  stochastic  effects.  The  model  for  vorticity  is  given  by 

^  +  J(tt,  q)  +  6SV2*  +  6m  =  A  sin  (2tt y  +  f(t)) 
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where  the  drive  f(t)  consists  of  a  periodic  component  and  additive  stochastic  terms.  The  drive  also 
introduces  multiple  scales.  The  results  of  numerically  approximating  such  a  scheme  are  given  as  the 
velocity  and  stream  function  for  generic  parameters: 


Velocity  u  and  Streamfunction  ¥ 


Fig.  1  Numerical  realization  of  the  stream  function  given  in  Eq.  1.  The  local  velocity 
field  is  depicted  by  arrows.  The  two  distinct  regions  are  counter-rotating. 


STOCHASTIC  MODEL  REDUCTION 

Due  to  the  high  dimensionality  and  multiple  scales  coupled  with  the  stochastic  forcing,  we  derived  a 
new  method  that  projects  the  model  onto  a  lower  dimensional  stochastic  model.  The  basic  problem  of 
model  reduction  in  deterministic  models  has  been  solved  using  center  manifold  theory.  However,  this 
will  NOT  work  when  stochastic  models  are  included.  Our  technique  is  a  novel  mathematical 
construction  that  correctly  captures  the  noise  in  the  lower  dimensional  model.  (E.  Forgoston  and  I.B. 
Schwartz,  Escape  Rates  in  a  Stochastic  Environment  with  Multiple  Scales,  arXiv:0809.1345  in  press 
SIAM  J.  Appl.  Dyn.  Systems  (2009)). 

We  applied  the  technique  to  the  simplest  following  problem: 
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2. 


y+fy I? 

£)fe=  F(x,  y)  =  (x-x3  -  y)+  (j)2 


Here  s  is  a  small  parameter  that  is  essentially  the  ration  of  the  slow  to  fast  time  scales  of  the  problem. 
The  terms  (|>i .  1=1,2,  are  stochastic  terms,  which  are  Gaussian  with  intensity  D.  For  small  s,  the 
dynamics  is  almost  one-dimensional.  The  results  are  shown  in  Fig.  2,  which  compares  the  theory  and 
simulations  for  different  noise  intensities,  D. 


Fig.  2.  Theory  (blue)  and  simulations  (black)  of  Eq.  2.  as  a  function  of  noise 
intensity.  The  y-axis  is  the  mean  escape  time  from  a  specified  region.  There  is  no 
control  implemented  here.  The  stochastic  manifold  reduction  was  used  to  create  a 
one-dimensional  mode  (theory  curve),  and  the  simulation  data  was  for  the  full 
system.  The  exponents  of  the  escape  time  are  given  by  the  slopes  of  the  curves,  and 

they  are  almost  identical. 

The  correct  noise  projection  using  stochastic  center  manifold  theory  predicts  maximal  probability 
regions  of  escape,  shown  in  Fig.  3,  whereas  the  usual  techniques  for  model  reduction  generate  wrong 
statistics. 
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Fig.  3.  Transition  probability  of  escape  is  shown  as  the  color  bar.  Higher 
probabilities  are  in  red.  The  optimal  path  of  escape  is  along  the  stochastic  center 
manifold  depicted  by  the  red  curve.  (See  arXiv:0809.1345  SIAM  J.  Appl.  Dyn. 

Systems  (2009)  for  details.). 

TRANSPORT  AND  UNCERTAINTY  FOR  STOCHASTIC  CONTROL 

For  the  transport  model  in  the  single  layer  ocean  model,  we  can  sample  the  dynamics  discretely  every 
period.  If  the  noise  added  is  a  random  variable,  r\  chosen  from  a  distribution,  v,  we  have  the  map  F: 
M-*M,  x  -*  F(x) .  The  density  evolution,  p,  may  be  computed  using  the  Stochastic  Frobenius-Perron 
(SFP)operator  (Billings,  L.  and  Schwartz  ,IB.  Identifying  almost  invariant  sets  in  stochastic  dynamical 
systems,  CHAOS  18,  art.  No  023122  2008.). 


Fig.  4  (Left  panel)  Uncertainty  regions  are  represented  by  the  histogram  colors, 
red  being  the  most  uncertain  points.  Uncertain  points  are  those  points  which  have 
a  non-zero  probability  of  escaping  from  A  to  B  or  vice  versa.  (Right  panel)  The 
brown  regions  are  almost  invariant  sets  which  are  computed  using  the  SFP 
described  above  in  the  text.  The  colored  regions  have  high  transition  probabilities 

going  from  one  basin  to  the  next. 

The  transport  from  one  region  to  the  next  allows  one  to  design  effective  monitoring  regions  for  control. 
For  example,  if  we  wish  to  maintain  a  glider  in  region  A  of  Fig.  4,  then  we  just  need  to  monitor  the 
uncertain  regions  to  maintain  the  dynamics  in  region  A.  This  is  the  basic  design  of  the  control  scheme. 
When  coupled  with  the  stochastic  cener  manifold  approaches,  it  works  exceedingly  well,  as  in  Fig.  5 
below. 
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Fig.  5  (Left  panel)  Increase  in  loitering  time  with  stochastic  control  for  single  layer 
stream  model.  Plotted  is  the  mean  escape  time  as  a  function  of  noise:  no  control 
(blue),  control  amplitude  0.025  (green),  control  actuation  0.06  (red).  Notice  the 
increase  in  slopes  with  greater  control  authority.  (Right  panel)  A  sample  of  a 
controlled  time  series.  The  control  actuations  are  in  red,  and  less  than  5  per  cent  of 

the  total  loitering  time. 

Figure  5  shows  that  with  even  modest  controls  that  are  infrequent,  we  can  achieve  exponentially 
longer  loitering  times  provided  we  can  specify  the  uncertainty  regions.  Because  our  method  is  general, 
uncertainty  specification  may  be  done  with  real  ocean  models  in  the  future,  when  coupled  with  data. 

MULTI-PARTICLE  SYSTEMS  EFFECTS  OF  NOISE  AND  DELAY 

We  considered  model  of  self-propelling  particles  interacting  through  pairwise  forces  in  presence  of 
noise  and  time  delay.  We  found  that  time  delay  in  communication  between  agents  induces  a  transition 
that  depends  on  the  size  of  the  coupling  amplitude,  and  the  transition  is  associated  with  a  derived  Hopf 
bifurcation.  Analytical  and  numerical  value  of  transitions  agree  very  well,  and  can  be  seen  in 
Forgoston,  E.  and  Schwartz  IB,  Delay-induced  instabilities  in  self-propelling  swarms,  Phys.  Rev.  E 
Rapid.  Comm  77  035203  (2008) 

ISSUES  AND  PLANS 

We  wish  to  continue  the  work,  and  unify  the  research  done  between  NRL  and  UCSB,  CUNY.  In 
addition,  some  of  the  control  work  on  the  single  layer  ocean  model  has  been  done  in  collaboration  with 
Montclair  State  (profs.  Lora  Billings  and  Phil  Yecko),  and  they  should  be  involved  formally  as  well. 
There  exist  several  issues  which  need  to  be  considered  in  order  to  push  the  optimal  control  of  gliders  in 
the  realm  of  3D  ocean  modeling,  and  they  are  itemized  below.  For  each  issue,  the  plans  subsection 
considers  a  viable  approach  to  complete  the  project. 

Issues 

1. Improve  stochastic  control  of  standard  stream  models 
2.Understanding  of  more  realistic  (harder)  models 

•Decomposition  into  coherent  structures  and  projected  models-Intermediate  goal 
•Analytic  stochastic  center  manifold  of  3D  PDE  ocean  models-Long  term  goal 
3. Incorporating  real  data  and  analysis  of  flow  methods 
4.Designs  towards  real  time  control  of  real  3D  gliders. 

Plans: 

1. Incorporate  optimal  and  stochastic  controls-UCSB,CUNY,  NRL 
2.Use  3D  layer  ocean  model  (Montclair) 
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3.Use  real  ocean  model  (NAVCOM)  using  data  assimilation  to  generate  dynamic  forces  on  a 
3D  glider. 

•Implement  3D  glider  model  (We  now  have  one.) 

•Using  data/models,  use  prehistory  and  FTLE  to  describe  dynamical  regions  and  escape 
probabilities. 

4. Implement  stochastic  optimal  control  scheme  in  3D  models. 

STATISTICAL  DATA 
Transitions 

None 

Related  Projects 

Design  of  Cooperative  Mobile  Agents  in  collaboration  with  Northwestern  university  Department  of 
Mechanical  Engineering, 
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